use privileged client in registry instead of user client when getting or creating signatures #16353

mfojtik · 2017-09-14T14:13:17Z

runcom · 2017-09-14T14:16:31Z

👍

runcom · 2017-09-14T14:19:33Z

mfojtik · 2017-09-14T14:44:23Z

@csrwng i was wrong, this should imagestreamimage and not imagesignatures (imagesignatures does not have 'get' method, just create and delete), registry reads the signatures by querying the imagestreamimage

runcom · 2017-09-14T14:47:57Z

(for the record we need this fix in 3.7, but I see it's alpha so this should go in anyway /cc @mrunalp)

csrwng · 2017-09-14T14:53:25Z

I can confirm that this worked for me on a test cri-o cluster.

mfojtik · 2017-09-14T14:58:14Z

@liggitt || @enj the registry has an signature endpoint that you can use to get signature... this endpoint is proxying the signature data from openshift (signatures are stored in openshift images...).

I believe that granting the image-puller ability to get the imagestreamimage to serve the signature content is fine, but want to double-check if you can think of any security implications.

This is the client call: https://github.com/openshift/origin/blob/master/pkg/dockerregistry/server/signaturedispatcher.go#L156

csrwng · 2017-09-14T15:01:28Z

pkg/cmd/server/bootstrappolicy/policy.go

@@ -533,6 +533,8 @@ func GetOpenshiftBootstrapClusterRoles() []rbac.ClusterRole {
 			Rules: []rbac.PolicyRule{
 				// pull images
 				rbac.NewRule("get").Groups(imageGroup, legacyImageGroup).Resources("imagestreams/layers").RuleOrDie(),
+				// read signatures


you could just add "imagestreamimages" to the resources in the rule above this one.

wanted to make it clear why that rule is there

enj · 2017-09-14T15:12:06Z

the registry has an signature endpoint that you can use to get signature... this endpoint is proxying the signature data from openshift (signatures are stored in openshift images...)

Er so what is preventing us from using that?

I believe that granting the image-puller ability to get the imagestreamimage to serve the signature content is fine, but want to double-check if you can think of any security implications.

So what extra things can image-puller see/do with this permission? I see that it can get the entire image, and not just the signatures. I just do not have enough context to say how much extra information you are leaking to get access to the signature data.

@openshift/sig-security

liggitt · 2017-09-14T15:17:36Z

pkg/cmd/server/bootstrappolicy/policy.go

@@ -533,6 +533,8 @@ func GetOpenshiftBootstrapClusterRoles() []rbac.ClusterRole {
 			Rules: []rbac.PolicyRule{
 				// pull images
 				rbac.NewRule("get").Groups(imageGroup, legacyImageGroup).Resources("imagestreams/layers").RuleOrDie(),
+				// read signatures
+				rbac.NewRule("get").Groups(imageGroup, legacyImageGroup).Resources("imagestreamimages").RuleOrDie(),


this role is intended to grant permissions to call the registry endpoints. the registry is already authorizing getting signatures based on get imagestreams/layers, have the registry's signature endpoint get signatures from the API using the registry's client, not the user's

mfojtik · 2017-09-14T15:35:13Z

@liggitt tested locally and it is working (it returns empty signatures)

liggitt · 2017-09-14T16:25:40Z

pkg/dockerregistry/server/signaturedispatcher.go

-	client, ok := userClientFrom(s.ctx)
-	if !ok {
-		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail("unable to get origin client"), w)
+	client, err := RegistryClientFrom(s.ctx).Client()


before we let the user create signatures via the registry, verifyImageSignatureAccess ensures they have create imagesignatures permission, so using the user client here would work. I'm not actually sure whether the registry has create imagesignatures permission by default

liggitt · 2017-09-14T16:25:48Z

pkg/dockerregistry/server/signaturedispatcher.go

-	client, ok := userClientFrom(s.ctx)
-	if !ok {
-		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail("unable to get origin client"), w)
+	client, err := RegistryClientFrom(s.ctx).Client()


this looks fine

liggitt · 2017-09-14T16:26:16Z

switching to use the registry client on get looks fine. I'm not sure about the switch for put

csrwng · 2017-09-14T17:03:04Z

The code in its current state worked for me on cri-o

jim-minter · 2017-09-14T18:06:03Z

/unassign jim-minter

giuseppe · 2017-09-14T21:08:26Z

pkg/dockerregistry/server/signaturedispatcher.go

-		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail("unable to get origin client"), w)
+	client, err := RegistryClientFrom(s.ctx).Client()
+	if err != nil {
+		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail(fmt.Sprintf("unable to get origin client", err)), w)


does the Sprintf miss the formatter?

yes, good catch, thx

giuseppe · 2017-09-14T21:09:56Z

pkg/dockerregistry/server/signaturedispatcher.go

-		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail("unable to get origin client"), w)
+	client, err := RegistryClientFrom(s.ctx).Client()
+	if err != nil {
+		s.handleError(s.ctx, errcode.ErrorCodeUnknown.WithDetail(fmt.Sprintf("unable to get origin client", err)), w)


simo5 · 2017-09-14T21:23:47Z

Do we have a test that confirms user's that have no access to adding signtures actually fail to add them ?
If not we MUST (as in RFCs) have such a test as part of this PR.

liggitt · 2017-09-16T23:57:22Z

# github.com/openshift/origin/pkg/dockerregistry/server
pkg/dockerregistry/server/signaturedispatcher.go:145: undefined: RegistryClientFrom
[ERROR] PID 1623: hack/lib/build/binaries.sh:228: `GOOS=${platform%/*} GOARCH=${platform##*/} go install -pkgdir "${OS_OUTPUT_PKGDIR}/${platform}" -tags "${OS_GOFLAGS_TAGS-} ${!platform_gotags_envvar:-}" -ldflags="${local_ldflags}" "${goflags[@]:+${goflags[@]}}" "${nonstatics[@]}"` exited with status 2.

mfojtik · 2017-09-17T18:16:10Z

@dmage seems like this was broken by #15796 can you please provide an advise how can one get privileged client in signaturedispatcher?

liggitt · 2017-09-17T18:23:00Z

pass registryClient to RegisterSignatureHandlers and hold a reference to it in the handler, rather than retrieve it from the context

mfojtik · 2017-09-17T18:24:46Z

@liggitt yeah, already working on that...

dmage · 2017-09-17T18:25:25Z

@mfojtik make RegisterSignatureHandler and SignatureDispatcher to be methods of (app *App). That will allow you use app.registryClient in SignatureDispatcher. You can put app.registryClient into signatureHandler.

mfojtik · 2017-09-17T19:10:19Z

@dmage done, unfortunately i have to plumb the unit tests as well... puzzled how they ever worked as the signaturedispatcher use the clientset and not openshift client.

dmage · 2017-09-17T19:17:03Z

@mfojtik don't forget to push your changes :)

mfojtik · 2017-09-17T19:18:05Z

@dmage once they compile...

mfojtik · 2017-09-17T19:43:39Z

@dmage ok, running out of options, but i'm getting signaturedispatcher_test.go:113: unexpected response status: 401 when running the TestSignatureGet, any clues?

mfojtik · 2017-09-17T19:46:10Z

sunday pebkac, nvmd.

mfojtik · 2017-09-17T19:49:39Z

pkg/dockerregistry/server/signaturedispatcher_test.go

@@ -57,6 +57,8 @@ func TestSignatureGet(t *testing.T) {
 		t.Fatal(err)
 	}

+	os.Setenv("OPENSHIFT_DEFAULT_REGISTRY", "localhost:5000")


@dmage for some strange reasons this is needed (at least on OSX) otherwise the NewApp() will fail... we should make this a field in openshift middleware config

Can you fix this as part of this PR? This is pretty wonky

@mfojtik yes, configuration check is moved to the app constructor. Adding a field in config will introduce yet another one environment variable for this value (we already have DEFAULT_REGISTRY_URL and OPENSHIFT_DEFAULT_REGISTRY). But I like this idea.

fixed in a separate commit

dmage · 2017-09-18T10:13:36Z

pkg/dockerregistry/server/repositoryconfig.go

@@ -74,6 +78,11 @@ func newRepositoryConfig(ctx context.Context, options map[string]interface{}) (r
 	} else {
 		context.GetLogger(ctx).Infof("DEPRECATED: %q is deprecated, use the %q instead", DockerRegistryURLEnvVar, OpenShiftDefaultRegistryEnvVar)
 	}
+	rc.registryAddr, err = getStringOption(DockerRegistryURLEnvVarOption, "dockerregistryurl", rc.registryAddr, options)


Only if rc.registryAddr has not filled yet.

i would say that having this in configuration file overrides all env vars?

Your code erases previous value no matter what. And environment variables usually overrides values in a config file.

dmage · 2017-09-18T10:15:22Z

pkg/dockerregistry/server/repositoryconfig.go

@@ -15,6 +15,10 @@ const (
 	// DEPRECATED: Use the OPENSHIFT_DEFAULT_REGISTRY instead.
 	DockerRegistryURLEnvVar = "DOCKER_REGISTRY_URL"

+	// DockerRegistryURLEnvVarOption is an optional environment that overrides the
+	// DOCKER_REGISTRY_URL.
+	DockerRegistryURLEnvVarOption = "REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_DOCKER_REGISTRY_URL"


s/DOCKER_REGISTRY_URL/DOCKERREGISTRYURL/

why do we need a second envvar for this?

dmage · 2017-09-18T10:18:43Z

pkg/dockerregistry/server/signaturedispatcher_test.go

-			"storage":    {{Name: "openshift"}},
+			"registry":   {{Name: "openshift", Options: configuration.Parameters{"dockerregistryurl": "localhost:5000"}}},
+			"repository": {{Name: "openshift", Options: configuration.Parameters{"dockerregistryurl": "localhost:5000"}}},
+			"storage":    {{Name: "openshift", Options: configuration.Parameters{"dockerregistryurl": "localhost:5000"}}},


this option is only for the repository middleware

mfojtik · 2017-09-18T13:53:19Z

@dmage comments addressed

mfojtik · 2017-09-18T14:39:08Z

/retest

dmage · 2017-09-18T14:40:35Z

/lgtm

openshift-merge-robot · 2017-09-18T14:40:55Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dmage, liggitt, mfojtik

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~pkg/dockerregistry/OWNERS~~ [dmage,liggitt,mfojtik]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

openshift-merge-robot · 2017-09-18T18:46:44Z

Automatic merge from submit-queue (batch tested with PRs 15834, 16321, 16353, 15298, 15433)

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 14, 2017

openshift-merge-robot assigned enj and jim-minter Sep 14, 2017

openshift-merge-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 14, 2017

openshift-ci-robot requested a review from giuseppe September 14, 2017 14:19

mfojtik force-pushed the allow-read-signatures branch from fa7a498 to afef3f6 Compare September 14, 2017 14:38

csrwng reviewed Sep 14, 2017

View reviewed changes

liggitt reviewed Sep 14, 2017

View reviewed changes

mfojtik force-pushed the allow-read-signatures branch from afef3f6 to 6d82a66 Compare September 14, 2017 15:19

openshift-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 14, 2017

mfojtik changed the title ~~allow image-puller role to read image signatures~~ use privileged client in registry instead of user client when getting or creating signatures Sep 14, 2017

mfojtik added this to the 3.7.0 milestone Sep 14, 2017

liggitt reviewed Sep 14, 2017

View reviewed changes

openshift-ci-robot unassigned jim-minter Sep 14, 2017

giuseppe reviewed Sep 14, 2017

View reviewed changes

mfojtik force-pushed the allow-read-signatures branch from 5606ec5 to 321715b Compare September 17, 2017 19:43

openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 17, 2017

registry: use the privileged client to get signatures

f7ec657

mfojtik force-pushed the allow-read-signatures branch from 321715b to f7ec657 Compare September 17, 2017 19:47

mfojtik commented Sep 17, 2017

View reviewed changes

dmage reviewed Sep 18, 2017

View reviewed changes

registry: add dockerregistryurl optional to middleware config

f8d7d28

mfojtik force-pushed the allow-read-signatures branch from c8e0063 to f8d7d28 Compare September 18, 2017 13:52

openshift-ci-robot assigned dmage Sep 18, 2017

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 18, 2017

openshift-merge-robot merged commit e4defee into openshift:master Sep 18, 2017

mfojtik deleted the allow-read-signatures branch September 5, 2018 21:07

use privileged client in registry instead of user client when getting or creating signatures #16353

use privileged client in registry instead of user client when getting or creating signatures #16353

Conversation

mfojtik commented Sep 14, 2017 • edited Loading

runcom commented Sep 14, 2017

runcom commented Sep 14, 2017

mfojtik commented Sep 14, 2017

runcom commented Sep 14, 2017

csrwng commented Sep 14, 2017

mfojtik commented Sep 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enj commented Sep 14, 2017

Choose a reason for hiding this comment

mfojtik commented Sep 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liggitt commented Sep 14, 2017

csrwng commented Sep 14, 2017

jim-minter commented Sep 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simo5 commented Sep 14, 2017

liggitt commented Sep 16, 2017

mfojtik commented Sep 17, 2017

liggitt commented Sep 17, 2017

mfojtik commented Sep 17, 2017

dmage commented Sep 17, 2017

mfojtik commented Sep 17, 2017

dmage commented Sep 17, 2017

mfojtik commented Sep 17, 2017

mfojtik commented Sep 17, 2017

mfojtik commented Sep 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfojtik commented Sep 18, 2017

mfojtik commented Sep 18, 2017

dmage commented Sep 18, 2017

openshift-merge-robot commented Sep 18, 2017

openshift-merge-robot commented Sep 18, 2017

mfojtik commented Sep 14, 2017 •

edited

Loading

mfojtik commented Sep 14, 2017 •

edited

Loading